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Introduction 


The  National  Training  Center  (NTC)  was  established  in  accordance  with  Army 
Regulation  (AR)  350-50  which  defines  NTC  missions  and  responsibilities.  The 
objectives  of  the  NTC  are  to: 

•  Provide  a  facility  where  heavy  battalion  task  forces,  controlling 
brigade  headquarters,  and  supporting  units  can  undergo  essential 
combined  arms  training  that  cannot  be  accomplished  at  home  stations 
due  to  physical  limitations  and  prohibitive  cost  of  providing  a 
realistic  training  environment. 

•  Gather  information  to  help  improve  doctrine,  tactics,  training 
system,  equipment  and  procedures.  This  information  also  assists  the 
Army  in  relating  resources  to  readiness. 

These  purposes  were  reiterated  in  the  Chief  of  Staff's  NTC  Policy 
Statement  in  September,  1984,  in  which  he  emphasized  the  need  and  the 
challenge  to  continue  the  tough,  successful  training  for  Battalion  Task 
Forces,  while  finding  ways  to  expand  NTC's  capabilities  to  promote 
innovation.  One  way  to  meet  this  challenge,  he  stated,  was  to  "develop  the 
NTC  range  instrumentation  and  associated  long-range  plans  to  permit  detailed 
analysis  and  feedback.  ...  We  must  all  work  together  to  harness  the  NTC's 
full  potential  and  spread  the  NTC  experience  throughout  the  total  Army." 

The  NTC  has  been  extremely  successful  in  meeting  its  training  goals. 

Unit3  have  been  exposed  to  the  most  realistic,  intense  training  environment 
ever  developed.  Faced  with  a  highly  trained  and  motivated  opposing  force 
(OPFOR),  continuous  training,  and  highly  dedicated  observer /controllers  (OCs), 
the  training  experience  which  units  receive  is  unparalleled. 

The  development  of  Lessons  Learned  has  been  a  slower  process.  According 
to  AR  350-50,  "The  training  environment  will  be  paramount  at  the  NTC." 

Primary  emphasis  has,  therefore,  been  placed  on  the  training  mission  of  the 
NTC.  In  August,  1985,  the  Center  for  Army  Lessons  Learned  (CALL)  was 
established  within  the  Combined  Arms  Training  Activity  (CATA).  CALL'S  mission 
is  to  provide  combat  relevant  Lessons  Learned  to  the  total  Army.  The  Army 
Research  Institute  (ARI)  has  a  letter  of  agreement  with  CATA  which  includes 
tasks  directly  relevant  to  this  mission.  Specifically,  ARI  has  agreed  to: 

•  Develop  methodology  for  the  use  of  NTC  findings  in  doctrine, 
organization,  equipment,  and  training  development. 

•  Develop  methods  to  improve  the  utility  and  quality  of  NTC  data. 

On  May  13-14,  1986,  the  first  meeting  of  a  planned  series  of  working 
groups  to  improve  the  quality  and  utility  of  NTC  data  was  held  at  ARI, 

Presidio  of  Monterey.  This  report  summarizes  the  results  of  that  meeting.  In 
addition,  this  report  includes  a  compilation  of  users'  needs  of  NTC  data  which 
have  been  gathered  over  a  period  of  several  years.  The  current  status  of  the 
NTC  data  collection  system  is  described  and  its  capability  is  compared  to 
users'  needs.  Requirements  for  meeting  discrepancies  are  outlined  and 
recommendations  for  improving  the  data  collection  system  are  made. 
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User  Needs 


Definition  of  User 


The  agencies  or  groups  interested  in  utilizing  NTC  data  sources  fall  into 
two  categories.  There  are  primary  users  of  the  raw  data  who  have  the 
capability  of  analyzing  that  data  and  putting  it  into  a  format  which  secondary 
users  can  interpret  and  utilize  for  decision  making. 

Among  the  primary  users  are  CATA,  the  research  community  (e.g.,  ARI, 

Arroyo  Center),  and  the  modeling  and  gaming  community  (e.g.,  TRADOC  Analysis 
Center).  The  analysts  at  the  NTC  are  also  primary  users  who  use  the  data  for 
their  After  Action  Reviews  (AARs)  and  Take  Home  Packages  (THPs). 

The  primary  users  put  the  data  into  a  format  which  is  usable  by  others. 
These  include  the  schools,  computer  assisted  battle  simulations,  trainers  at 
home  station,  and  policy  makers  at  Department  of  the  Army,  Forces  Command, 

Army  Materiel  Command,  and  Training  and  Doctrine  Command. 


Initial  Working  Group  Meeting 

Purpose.  With  the  establishment  of  the  Lessons  Learned  program,  the 
applications  of  NTC  data  and  the  number  of  agencies  and  individuals  involved 
in  data  collection  and  analysis  have  increased.  Each  agency  has  a  partial 
view  of  the  data  collection  and  processing  system.  ARI  initiated  the 
establishment  of  a  working  group  to,  first,  share  information  on  current 
activities  and  problems  or  questions.  This  exchange  of  information  was 
intended  to  provide  a  common  broad-base  description  of  issues. 

The  second  purpose  was  to  develop  recommendations  for  near-term  and 
longer-term  approaches  to  improving  the  quality  and  quantity  of  information 
from  the  NTC.  With  these  goals  in  mind,  CATA  invited  the  participants  to 
attend  the  May  1 3—1 ^ •  1986,  meeting  held  by  ARI,  at  its  Presidio  of  Monterey 
Field  Unit.  Appendix  A  gives  an  overview  of  ARI's  NTC  Research  Program,  as 
briefed  at  this  meeting.  ARI  has  resources  in  all  of  its'  three  laboratories 
working  on  NTC-related  projects  and  a  number  of  research  products  have  been 
and  are  being  developed. 

Participants.  The  following  agencies  were  represented  at  the  meeting  (see 
Appendix  B  for  list  of  individuals  in  attendance): 

•  Combined  Army  Training  Activity 

•  TRADOC  Analysis  Center 

•  AMEX  Corporation 

•  Army  Training  Support  Center 

•  Arroyo  Center 

•  Army  Training  Board 

•  Combat  Development  Experimentation  Command 

•  United  States  Army  Infantry  Center 

•  The  BDM  Corporation 
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Identification  of  Needs 


After  a  series  of  briefings  (which  are  integrated  into  this  report  and 
included  as  appendixes),  the  working  group  generated  a  list  of  data  of 
interest  to  them.  This  list  was  divided  into  near  and  longer  term  based  on 
the  difficulty  of  obtaining  the  data.  The  near-term  data  were  prioritized 
into  three  categories,  based  on  the  value  of  the  information  and  the  amount  of 
work  required  for  NTC  personnel  to  collect  and  provide  it.  Near-term,  high 
priority  data  needs  are  included  in  the  Recommendations  section  of  this 
report.  Longer-term  or  lower-priority  needs  are  at  Appendix  C. 

This  meeting  was  the  most  recent  of  many  efforts  to  collect  similar 
information.  In  1982,  the  Combined  Arms  Center  (CAC)  solicited  input  on 
requirements  for  an  NTC  Data  Base  Library.  Host  of  the  responses  gave 
detailed  lists  of  the  type  of  data  various  agencies  would  be  interested  in. 
Responses  are  at  Appendix  D. 

The  information  at  Appendix  D  was  distilled  by  CAC  through  an  NTC  Combat 
Mission  Analysis.  They  developed  1500  questions  to  be  answered  for  each  unit 
(see  Appendix  £). 

In  1985,  an  NTC  Firing  Data  Workshop  was  held  to  examine  need3  which  could 
be  met  with  NTC  data.  Notes  on  this  workshop  are  at  Appendix  F.  ARI's 
resident  contractor  at  the  Presidio  of  Monterey  Field  Unit,  BDM,  also 
generated  a  list  of  potential  research  issues  which  might  be  addressed.  These 
are  categorized  by  Operating  System  and  are  at  Appendix  G. 

In  November,  1985,  CATA  hosted  a  NTC  Lessons  Learned  Workshop  to  extract 
and  disseminate  lessons  learned  and  increase  awareness  regarding  the  agencies 
available  to  collect  data,  how  they  collect  it,  and  what  they  produce.  The 
roster  of  those  in  attendance  is  at  Appendix  H.  Most  recently,  the  NTC  was 
used  to  collect  data  for  testimony  to  Congress  on  the  Bradley  Fighting 
Vehicle.  The  type  of  data  which  was  collected  and  which  would  also  be  of 
value  for  other  training  applications  is  at  Appendix  I. 

A  review  of  these  appendixes  makes  it  clear  that  there  are  a  magnitude  of 
potential  uses  of  NTC  data.  The  following  examples  of  questions  which  could 
potentially  be  answered  in  each  of  the  categories  specified  in  AR  350-50 
demonstrate  the  breadth  of  relevant  issues. 

Doctrine. 


•  What  problems  are  presented  by  the  J-series  table  of  organization  and 
equipment  (T04E),  particularly  with  dismounted  infantry? 

•  Where  should  Command  Groups  be  positioned  on  the  battlefield? 

•  How  effective  is  air  defense  early  warning  doctrine? 

•  Does  current  doctrine  adequately  show  units  how  to  attack  Soviet 
strongpoint-type  defense? 
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Tactics 


•  What  is  the  contribution  of  Infantry  anti-tank  weapon  systems  to  the 
close-in  anti-armor  battle? 

•  To  what  extent  is  combat  power  concentrated  in  conducting  a 
deliberate  attack? 

•  How  can  mortars  be  effectively  utilized  in  battle? 

•  Was  the  amount  of  terrain  offering  cover  and  concealment  considered? 

Training  system. 

•  How  can  the  effectiveness  of  tank  and  tube-launched,  optically- 
tracked,  wire-guided  missile  system  (TOW)  crews  be  improved? 

•  To  what  extent  do  inaccurate  calls  for  fire  cause  direct  fire 
fratricides? 

•  To  what  extent  do  task  forces  employ  counterattacks  in  the  defense? 

•  Do  engineers  employ  minefield  equipment  and  techniques  effectively  in 
obstacle  construction? 

Equipment. 

•  How  does  the  Bradley  Fighting  vehicle  compare  in  performance  to  the 
M-113? 

•  Are  current  battalion  communications  equipment  packages  capable  of 
surviving  on  the  battlefield? 

•  What  was  the  average  down  time  for  radios  turned  in  for  maintenance? 

•  What  is  the  contribution  of  VIPER  and  DRAGON  to  the  anti-armor  fight? 

Procedures. 


•  How  effective  are  cross-attached  companies  as  compared  to  pure 
companies? 

•  How  can  the  fire  support  officer  (FSO)  best  synchronize  fires? 

•  What  is  the  most  effective  procedure  for  passing  early  warning 
information? 

•  Do  units  react  in  accordance  with  the  mission-oriented  protection 
posture  (MOPP)  level  designated  by  the  commander? 

Many  of  these  issues  could  potentially  be  answered  with  instrumented  data, 
e.g.,  position  location  and  firing  data.  Others  would  require  observational 
data  or  information  from  the  communication  nets.  It  is,  therefore,  important 
to  look  at  the  current  data  collection  system  to  determine  its  capability. 


Current  System 


There  are  three  primary  sources  of  NTC  data  available  to  potential 
users:  the  instrumentation  system,  Take  Home  Packages  (THPs),  and  40-channel 
communication  tapes  recorded  during  rotations.  Policies  and  procedures  at  the 
NTC  are  in  a  continual  state  of  evolution,  so  this  report  will  discuss  the 
capability  of  the  current  system,  as  known.  Each  of  the  three  sources  of  NTC 
data  will  be  described  and  discussed.  The  actual  use  of  the  instrumentation 
system  differs  from  its  design  and  potential  utility,  so  these  two  topics  will 
be  discussed  separately. 


Instrumentation  System 

Appendix  J  contains  the  briefing  materials  presented  at  the  initial 
working  group  meeting  by  AMEX,  Inc.,  the  developers  of  the  instrumentation 
system.  Two  subsystems  are  principally  involved  in  the  digital  data 
collection.  The  Range  Data  Measurement  System  (RDMS)  collects  raw  field 
data.  Much  of  this  data  is  transmitted  to  the  Core  Instrumentation  Subsystem 
(CIS),  where  data  can  also  be  input  manually. 

RDMS  data.  The  RDMS  collects  data  on  trigger  pull  (fire  event),  laser 
illumination  (pairing),  microphone  key  pressing,  and  the  raw  data  from  which 
position/location  is  derived  for  instrumented  players.  During  Live  Fire 
Exercises,  data  on  target  status  (up,  down,  hit)  is  also  collected.  The 
capacity  of  the  system  is  limited  so  there  are  a  large  number  of  uninstru¬ 
mented  players  for  which  position  and  event  data  cannot  be  collected.  In 
addition,  none  of  the  dismounted  personnel  are  instrumented.  For  example,  in 
a  sample  of  ten  defensive  missions,  an  average  of  27%  of  the  players  were  not 
instrumented.  The  percent  uninstrumented  ranged  from  a  low  of  17%  to  a  high 
of  38%. 

According  to  Briscoe  (1986): 

RDMS  data  are  as  accurate  as  the  reliability  of  the  collecting/ 
transmitting  hardware  allows.  The  complexity  of  NTC  field  instrumentation 
is  conducive  to  a  wide  variety  of  error-producing  conditions,  including: 

(1)  Spurious  radio  frequency  (RF)  transmissions,  leading  to 
erroneous  events; 

(2)  "Noisy"  laser  sensors  which  generate  spurious  and/or  inaccurate 
pairing  events; 

(3)  Hardware/electronic  player  instrumentation  problems  leading  to 
loss  or  duplication  of  valid  events,  and  the  generation  of  invalid  events; 
and 


(4)  Coverage  problems  resulting  in  the  loss  of  track  of  instrumented 
vehicles  and  the  corresponding  loss  of  position/location  and  event 
retrieval  capability. 
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Even  in  the  case  of  perfect  hardware  performance,  it  is  possible  for 
errors  to  be  introduced  by  faulty  initialization.  If  the  proper  B-unit  code 
is  not  associated  with  the  right  player  identification,  incoming  events  will 
be  improperly  assigned  or  may  be  deleted  as  invalid.  Such  problems  can 
quickly  lead  to  a  serious  loss  of  data  integrity. 

MILES.  The  force-on-force  exercises  use  laser-based,  engagement 
simulation  technology.  MILES  (Multiple  Integrated  Laser  Engagement  System)  is 
used  on  all  principal  weapons  and  results  are  assessed  when  a  weapon  fires  and 
the  MILES  laser  hits  a  target.  While  this  system  provides  a  high  degree  of 
realism  to  combat  training,  it  is  not  without  some  deficiencies.  For  example, 
the  laser  does  not  penetrate  smoke  and  a  target  which  may  be  sighted  with  a 
thermal  sight  could  not  be  "killed"  with  MILES. 

Another  problem  is  that  in  order  for  a  TOW  to  kill  a  target,  it  has  to 
track  the  target  for  15  seconds.  This  effectively  simulates  the  requirements 
for  targets  at  long  ranges,  but  not  short  ranges.  Therefore,  a  TOW  which 
tracked  a  target  for  less  than  15  seconds  at  short  range  would  not  be  counted 
as  a  kill,  even  though  in  actual  combat  the  length  of  time  may  have  been 
sufficient.  MILES  also  does  not  compensate  for  differences  in  requirements  to 
range  to  target  or  to  employ  a  lead  angle  by  actual  weapon  systems.  The  laser 
technology  instantaneously  hits  the  target  whereas  actual  ammunition  would 
take  some  period  of  time. 

CIS  data.  The  CIS  logs  data  in  real-time  to  provide  the  primary  archival 
source  of  NTC  data.  It  also  supports  pre-exercise  initialization  in  which 
player  information,  control  measures,  task  force  organization,  live  fire 
scenarios,  and  pre-planned  artillery  is  entered.  The  CIS  also  pairs  firers 
with  targets  and  calculates  statistical  measures. 

According  to  Briscoe  (1986); 

It  should  be  noted  that  many  of  the  elements  logged  from  the  CIS  are 

manually  input  data  elements.  Reliability  of  manual  data  depends  upon  the 

accuracy  of  the  personnel  entering  the  data  and  the  verification 

procedures  that  are  employed,  such  as  proofreading  and  consistency  checks. 

Digitized  history  database  development.  Appendix  K  shows  the  list  of 
statistical  displays  that  the  software  is  capable  oi  supporting.  Fobes  (1984) 
and  Science  Applications,  Inc.  documentation  give  detailed  descriptions  of 
each  of  these  displays.  These  data  are  displayed  for  a  mission  on  a  VT125 
Graphics  Monitor  when  a  specific  history  tape  i3  loaded.  This  means  that  data 
in  these  tables  can  only  be  collected  on  a  mission  by  mission  basis.  The 
primary  users  of  these  displays  are  the  analysts  who  prepare  the  AARs. 

Science  Applications,  Inc.  developed  software  for  ARI  intended  to 
translate  the  display  data  into  a  format  which  would  facilitate  analysis  with 
INGRES,  a  relational  data  base  management  system.  Sixty-one  INGRES  tables  are 
created  for  each  translated  history  segment  from  an  NTC  rotation.  A 
description  of  each  of  these  tables  is  at  Appendix  L. 

Instrumented  database  status.  Software  development  to  translate  mission 
data  into  INGRES  tables  is  still  on-going.  Among  the  remaining  problems,  for 
example,  are  the  following.  Position/location  data  are  critical  to  most 
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research  questions.  However,  the  software  takes  a  tremendous  amount  of  time 
(approximately  one  hour  of  position/location  data  takes  an  hour  of  computer 
processing  time)  to  process  these  data,  so  until  new  software  is  developed  to 
more  efficiently  handle  the  data,  translation  of  position/location  data  will 
be  limited.  Also,  the  current  process  creates  an  INGRES  database  for  each 
mission  segment.  Research  clearly  requires  being  able  to  aggregate  data 
across  missions,  but  this  is  not  easily  done.  The  current  database  is 
"inconvenient  to  use,  is  massively  redundant,  and  contains  superfluous  data" 
(Briscoe,  1986). 

In  addition,  there  is  a  great  deal  of  missing  data  which  are  of  potential 
research  value.  On  a  sample  of  71  missions,  the  number  of  empty  tables  ranged 
from  13  to  44  of  the  61  tables.  Only  an  average  of  35-36  tables  contained  any 
data.  Even  those  that  contain  data  may  be  incomplete.  For  example,  the  table 
SEGSUMRATING  is  supposed  to  contain  segment  summary  ratings  provided  by  key 
training  operations  group  personnel  (i.e.,  TAFO,  EMCO,  and  Chief  OC)  at  the 
end  of  the  segment.  The  collective  judgement  of  this  group  on  the  performance 
of  the  battalion  is  expressed  in  numbers  from  0-9,  using  the  following 
scheme:  0  =  no  observation;  1  =  very  poor;  3  =  poor  performance;  5  =  nominal 
expected  performance;  7  =  good  performance;  and  9  =  excellent  performance. 
These  ratings  are  to  be  made  on  Overall  Exercise  Effectiveness;  Target 
Acquisition;  Maneuver;  Fire;  Communication;  Command  and  Control,  and  Logistics 
and  Administration.  These  data,  although  subjective,  have  potential  research 
value.  However,  this  table  contains  only  the  date  and  time  the  mission 
ended.  All  ratings  are  missing  in  the  sample  of  missions  which  were  examined. 

There  are  two  sources  of  data  for  the  tables:  the  field  instrumentation 
system  and  operator-entered  inputs.  The  tables  which  require  manually  entered 
data  are  empty  or  sparse.  Thirty-three  of  the  61  tables  require  some  or  all 
data  to  be  manually  entered.  It  is  clear,  therefore,  that  the  current  system 
could  support  the  collection  of  a  great  deal  more  data  than  is  currently 
gathered.  The  utility  of  the  data  which  is  currently  being  archived  is  only 
marginally  useful  for  research  on  NTC  performance,  but  the  capability 
obviously  exists  for  significant  improvement. 

One  problem  which  has  impeded  the  database  development  process  is  a  lack 
of  documentation  of  the  SAI  software.  It  would  be  advantageous,  for  example, 
to  have  the  format  of  the  CIS  logs.  Without  documentation,  it  is  impossible 
to  know  if  the  CIS  tape  contains  everything  on  the  RDMS  tape. 

ARI  has  recently  developed  software  that  allows  for  reading  the  RDMS  log 
tapes.  The  RDMS  might  contain  additional  information  of  importance  for  off¬ 
line  analyses,  e.g.,  the  MILES  code  for  the  firing  weapon  for  each  vehicle 
which  has  been  killed.  This  information  might  also  be  on  the  CIS  tape,  but 
this  has  not  yet  been  determined.  The  consistency  between  the  CIS  log  and  the 
RDMS  log  is  currently  being  examined.  It  may  be  possible  to  extract 
additional  relevant  information  from  the  RDMS  tapes  for  research  purposes. 
Appendix  M  contains  working  group  briefing  slides  on  the  status  of  the  digital 
data  analysis  and  data  base  development. 
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Take  Home  Packages 


Appendix  N  contains  briefing  slides  from  the  initial  working  group  meeting 
which  describe  an  analysis  of  the  Maneuver  Performance  Trends  section  of  26 
THPs  from  1984  and  1985.  The  THPs  potentially  serve  two  purposes:  to  guide 
home  station  training  and  for  development  of  Lessons  Learned  across 
battalions. 

The  results  of  this  analysis  indicate  that  because  the  format  of  the  THPs 
is  not  related  to  the  ARTEP,  it  may  be  difficult  to  translate  the  Performance 
Trends  into  training  guidance  for  home  station.  Due  to  the  lack  of 
consistency  within  and  among  OC  training  teams  regarding  what  they  observe, 
the  data  is  also  of  little  value  for  deriving  Lessons  Learned.  Some  trends 
were  identified,  but  of  the  113  issues,  relatively  few  were  commented  upon 
consistently  enough  to  draw  conclusions. 

The  battalion  THPs  also  contain  After  Action  Review  (AAR)  summaries  for 
each  mission.  These  data  were  not  analyzed  r  id  are  potentially  of  value. 
Appendix  0  contains  briefing  slides  describing  an  analysis  of  this  section  of 
the  company  THPs  for  leadership  Lessons  Learned.  Again,  a  great  deal  of 
variation  was  found  in  the  amount,  quality,  and  format  of  information.  The 
key  to  THP  data  being  of  any  value  for  Lessons  Learned  is  standardization  of 
observations. 


Communication  Tapes 

A  recent  analysis  of  the  research  potential  of  the  40-channel  communica¬ 
tion  tapes  found  that  the  tapes  are  a  rich  source  of  detail  and  essential 
contextual  information  (see  Appendix  P).  They  can  be  synchronized  with  the 
workstation  display  of  the  battle  which  greatly  increases  understanding  of 
what  is  taking  place.  The  OC  control  nets  probably  provide  the  best  source  of 
data. 

The  major  problem  with  using  the  tapes  for  research  purposes  i3  that  it 
is,  currently,  a  very  labor-intensive  process  considering  that,  with  40 
communication  channels,  one  rotation  is  the  equivalent  of  1280  days  of 
recordings.  In  addition,  there  is  a  problem  with  override  of  channels. 
Electronic  methods  of  processing  the  information  would  not  be  able  to 
determine  whether  a  transmission  was  a  legitimate  one  or  if  it  was  coming 
through  from  another  net. 

There  is  a  large  discrepancy  between  the  number  of  transmissions  actually 
counted  in  listening  to  the  tapes  and  the  number  of  transmissions  reported  in 
the  THPs.  The  THPs  appear  to  report  far  fewer  transmissions  than  were  found 
on  the  tapes.  This  makes  the  THP  communication  data  misleading  and 
essentially  worthless.  This  disparity  needs  to  be  eliminated. 


Data  Availability 

Appendix  Q  contains  the  briefing  material  describing  current  and  planned 
NTC  data  acquisition  and  storage.  ARI  currently  has  digital  tapes  from 
August,  1983  to  the  present;  most  of  the  THPs  from  January,  1982  to  January, 
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1986;  some  operation  plans;  and  a  few  AAR  video  tapes  and  unit  After  Action 
Reports. 

Coordination  has  recently  been  accomplished  with  the  NTC  Operations  Group 
to  ensure  routine  acquisition  of  materials,  including  communication  tapes, 
exercise  control  logs,  and  operation  orders  by  ARI.  This  clearly  will  result 
in  a  requirement  for  storage  space.  To  reduce  this  requirement,  consideration 
is  being  given  to  transmitting  the  THPs  on  floppy  disks  and  to  compressing  the 
communication  tapes. 


Discrepancy  Between  User  Needs  and  Current  System 


The  existence  and  the  magnitude  of  potential  uses  of  data  for  Lessons 
Learned  from  the  NTC  was  clearly  demonstrated  in  the  initial  working  group 
meeting,  as  well  as  by  Appendixes  C  through  I  of  this  report.  The  questions 
raised  by  these  sources  cannot  be  answered  with  the  data  currently  archived. 
The  reasons  the  data  are  insufficient  to  answer  such  questions  include; 

•  The  lack  of  standardization  regarding  observations  made/recorded  by 
the  OCs.  Standardization  allows  quantification  of  information  and 
statistical  manipulation  and  permits  interpretation  of  results. 

•  The  large  amount  of  data  which  must  be  manually  entered  which  is 
missing.  Much  of  these  data  would  be  valuable  for  research  and 
analysis. 

•  Limitations  on  the  number  of  players  which  can  be  instrumented.  The 
current  system  supports  500  instrumented  players.  In  reality, 
usually  only  400  to  425  players  are  actually  instrumented.  This 
total  must  include  not  only  battalion  task  forces,  but  0PF0R,  OCs, 
and  brigade  players,  so  some  percentage  of  players  are  not 
instrumented  and,  therefore,  not  tracked.  To  the  extent  that  there 
is  any  systematic  rationale  behind  decisions  on  who  to  instrument, 
such  as  the  fact  that  no  dismounted  personnel  are  ever  instrumented, 
the  data  which  are  collected  may  not  be  representative.  There  is 
currently  a  Statement  of  Work  for  a  contract  to  increase  the  system 
to  1000  players. 

•  Coverage  problems.  Even  if  a  vehicle  is  instrumented,  the  terrain  at 
the  NTC  can  mask  it  so  that  its  current  position/location  cannot  be 
tracked.  This,  again,  results  in  missing  data.  If  the  missing  data 
were  random,  it  might  not  be  a  major  problem.  However,  if,  for 
example,  there  was  a  research  question  regarding  the  performance  of 
tanks  in  the  defense  and  the  better  performing  tanks  were  more 
effectively  hidden,  any  data  collected  on  the  less-well-hidden  tanks 
would  not  be  representative.  In  a  sample  of  ten  defensive  missions, 
for  example,  there  were  a  total  of  327  instrumented  tanks.  Of  these, 
12  percent  were  never  tracked  and  an  additional  6  percent  were  only 
partially  tracked. 
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•  Electronic  hardware  problems  leading  to  spurious,  inaccurate,  or 
duplication  of  events. 

•  The  fact  that  a  research  data  base  does  not  exist.  The  current 
structure  of  the  software  requires  that  data  be  accessed  mission  by 
mission  rather  than  across  missions.  Within  the  next  six  months  to 
one  year,  the  data  base  will  be  redesigned.  In  addition,  only  a 
small  part  of  the  information  required  for  analysis  and  interpreta¬ 
tion  is  in  digital  form.  Merging  of  data  from  multiple  sources  is 
manpower  and  time  intensive. 

The  above  issues  differ  in  terms  of  the  degree  to  which  they  can  be 
resolved.  The  next  section  of  this  report  discusses  requirements  for 
improving  the  quality  of  data  from  the  NTC  to  support  the  development  of 
Lessons  Learned. 


Requirements 


Procedural 


The  achievement  of  inter-rater  reliability  among  the  OCs  is  critical  for 
development  of  Lessons  Learned.  This  is  an  evolving  process  and  needs  to 
involve  training  programs.  The  system  which  is  now  in  place  was  designed  for 
training  and  has  been  very  effective  for  that  purpose.  However,  a  system 
which  will  support  both  training  and  generation  of  Lessons  Learned  i3 
necessary.  Appendix  R  contains  briefing  materials  on  OC  procedures.  It  is 
clear  that  the  OCs  are  extremely  busy.  OCs  are  very  competent  and  dedicated, 
but  they  v'“ed  tools  to  assist  them  in  making  systematic  observations,  so  that 
they  can  -work  smarter,  not  harder."  ARI  has  begun  to  assist  in  this  process 
within  the  CIS.  Training  materials  and  SOPs  have  been  and  are  being  developed 
for  their  use  (see  Appendix  S). 

It  is  clearly  necessary  to  determine  which  aspects  of  performance  need  to 
be  systematically  observed  and  measured.  Careful  analysis  is  needed  which 
will  consider  priorities  for  information,  as  well  as  what  is  desirable.  The 
ground  rules  regarding  what  needs  to  be  observed  have  to  be  standardized  and 
OCs  need  to  be  re-calibrated  after  they  have  been  in  the  field  for  a  while. 

ARI  is  currently  working  to  design  and  develop  a  system  for  measuring  and 
interpreting  the  effectiveness  of  unit  performance  at  the  NTC.  As  part  of 
this  effort,  an  NTC-specific  ARTEP  and  NTC  observer  guides  are  being 
developed.  Briefing  materials  describing  this  effort  are  at  Appendix  T. 


Hardware 


One  tool  which  would  be  of  great  utility  to  the  OCs  is  the  Electronic 
Clipboard.  Briefing  materials  on  this  are  at  Appendix  U.  This  piece  of 
equipment  would  allow  for  downloading  of  performance  observation  guides  into 
the  hand-held  clipboard.  Data  on  performance  would  be  input  into  the 
clipboard  by  the  OCs  in  the  field.  The  digital  data  in  the  clipboard  could 
then  be  radioed  to  the  CIS  or  uploaded  by  an  RS232  connector  directly  into  the 
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CIS.  This  innovation  will  increase  the  reliability  of  observations  by 
providing  standard  observer  guidance  cues  and  automation  for  the  recording  and 
processing  of  performance  ratings.  This  will  assist  the  OCs  in  doing  their 
jobs  rather  than  adding  to  their  responsibilities,  and  provide  improved 
information  for  AARs  and  THPs,  and  the  research  data  base. 

The  Electronic  Clipboard  is  also  one  of  the  components  of  the  Integrated 
Training  Management  System  which  is  being  developed  in  the  9th  Infantry 
Division.  This  division-level  system  is  being  developed  to  improve  the 
ability  of  units  to  prioritize,  resource,  conduct,  and  evaluate  training 
through  management  automation.  It  takes  into  account  training  priorities  set 
by  leaders;  prerequisite  relationships  among  individual  and  collective  tasks; 
availability  of  resources;  and  unit  training  needs.  A  prototype  system  has 
already  been  developed  by  ARI  and  successfully  tested  in  a  field  artillery 
battalion  of  the  9  ID  (MTZ).  This  system  has  great  potential  for  integrating 
NTC  experiences  with  home  station  training. 

Related  to  this  is  a  planned  ARI  contract  on  the  determinants  of  effective 
performance  at  the  NTC.  The  purpose  of  this  research  will  be  to  improve  unit 
training  through  better  integration  between  NTC  and  home  station  and  to 
determine  what  unit  training  management  practices,  organizational  processes, 
leadership  behavior,  and  individual  abilities  have  an  impact  on  performance  at 
the  NTC.  Briefing  materials  describing  this  effort  are  at  Appendix  V. 


Software 


As  previously  noted,  there  is  currently  a  Statement  of  Work  which  solicits 
proposals  to  upgrade  the  current  NTC  software.  Appendix  W  contains  briefing 
slides  and  a  memorandum  on  a  meeting  to  validate  the  requirements  of  this 
Statement  of  Work. 

The  upgraded  system  will  be  able  to  track  1000  players  instead  of  500.  It 
will  incorporate  new  nucler,  biological,  chemical  (NBC)  control  measures.  Air 
Defense,  Helicopters,  and  Air  Force  jet  aircraft  will  be  instrumented  with 
MILES  and  will  be  able  to  be  tracked  and  displayed.  It  will  automatically 
collect  tactical  fire  direction  system  (TACFIRE)  messages  and  have  increased 
fire  mission  storage. 

A  major  improvement  is  the  ability  to  assign  each  player  an  identification 
code.  Currently,  pairings  are  established  by  time  coincidence  and  player 
identifications  are  made  based  on  position  at  the  time  a  firing  event 
occurs.  The  new  system  provides  identification  of  the  firing  vehicle  and 
weapon  fire  for  each  MILES  hit/near  miss/kill  event.  It  will  provide 
graphical  display  of  individual  weapons  systems/player  unit  and  hit/near 
miss/kill  data.  Obtaining  player  and  crew  identification  was  identified  by 
the  working  group  as  a  medium  priority,  near-term  data  collection  need.  It 
would  appear  that  this  solicitation  will  supply  this  information. 

The  SOW  also  provides  delivery  of  the  NTC  workstations  to  home  station 
units.  This  will  give  units  the  capability  to  replay  NTC  exercises.  An 
upgrade  of  the  video  and  audio  capabilities  is  also  provided  for.  These 
improvements  are  to  be  implemented  incrementally,  but  should  be  completed  two 
years  from  award  of  contract. 


Recommendations 


The  recommendations  in  this  section  were  developed  from  the  working  group 
meeting  held  in  May,  1986.  The  most  immediate  requirement  is  that  ARI  be 
provided  detailed  information  on  current  procedures  used  at  the  NTC.  The 
training  environment  at  the  NTC  is  in  a  state  of  constant  evolution.  ARI  will 
have  a  researcher  in  residence  at  the  NTC  in  July,  1986,  which  will  facilitate 
tracking  of  procedures. 

It  is  also  important  that  a  concerted  effort  be  made  to  get  information 
from  units  regarding  their  needs  in  the  AARs  and  THPs.  This  will  allow  a 
better  integration  of  training  and  Lessons  Learned  needs.  It  would  be  of 
value  to  know  if  and  how  units  use  the  information  provided  to  them  and  what 
additional  data  would  be  of  interest.  This  research  effort  is  now  in  the 
planning  stage  by  ARI. 

It  is  important  to  ensure  that  the  NTC  retains  the  tapes  of  the  40-channel 
communication  networks  recorded  for  each  rotation.  The  cost  of  the  tapes  has 
led  to  reusing  them  after  approximately  six  months.  One  of  the  briefings  in 
the  working  group  meeting  described  an  evaluation  of  the  research  potential  of 
the  tapes  and  recommended  that  they  be  retained.  The  consensus  of  the  group 
was  that  the  cost  of  the  tapes  within  the  context  of  the  seven  million  dollar 
cost  of  the  rotation  was  relatively  minor.  Recent  coordination  with  the  NTC 
has  supported  this  action. 

There  is  currently  information  being  collected  at  the  NTC  which  is  not 
being  provided  to  ARI.  Evidently,  information  on  indirect  fire  currently 
exists  in  a  paper  form,  but  is  not  retained  after  a  rotation.  Also,  .ivision, 
brigade,  battalion,  and  OPFOR  operations  orders  and  graphics  for  each  mission 
exist,  but  are  not  collected  or  retained.  Army  Material  Systems  Analysis 
Agency  (AMSAA)  currently  collects  data  on  Combat  Service  Support  and 
operational  readiness  to  provide  Lessons  Learned  to  the  Materiel  Readiness 
Support  Agency.  This  data  should  be  relatively  easy  to  obtain  to  assess  its 
utility  for  other  analyses. 

The  current  software  has  the  capacity  to  collect  free  format  messages  with 
a  maximum  length  of  three  lines  of  text  of  80  characters  per  line.  If  the  OCs 
in  the  field  would  have  the  CIS  analysts  routinely  enter  information  with 
important  research  potential,  it  would  be  a  relatively  simple  method  of 
collecting  data  for  Lessons  Learned.  Of  high  priority  to  the  working  group 
was  to  use  this  capability  to  collect  information  on  the  number  or  percent  of 
friendly  and  enemy  kills,  by  weapon  system,  after  every  battle.  The  OCs  know 
this  information,  but  it  is  not  collected  in  a  systematic  fashion  and  was  of 
great  interest  to  all  of  the  meeting  participants. 

Also  of  high  priority  for  collection  with  the  free  format  message 
capability  was  entering  and  time  tagging  verbal  fragmentary  orders  (FRAGOs) 
during  the  battle.  Also  of  interest,  were  free  format  messages  on: 

•  smoke  conditions  and  major  weather  changes; 

•  dismounted  infantry  positions  and  casualties  held  some  interest  but 
were  regarded  as  a  difficult  data  collection  burden  for  the  OCs; 


•  start  and  end  times  of  the  battle  would  be  useful; 

•  the  reasons  for  OC  kills  and  resurrections  are  very  important. 

This  method  could  also  be  used  to  collect  information  on  informal, 
spontaneous  organizations  that  emerge  on  the  battlefield.  However,  this  was 
of  bottom  priority  for  the  group.  It  is  important  to  emphasize  that  if 
procedures  are  implemented  to  collect  any  or  all  of  this  data  that  it  must  be 
done  in  a  consistent  fashion.  Partial  or  inconsistent  use  of  free  format 
messages  would  make  the  data  uninterpretable. 

Another  useful  set  of  data  which  could  be  supported  by  the  current 
software  is  the  header  data  for  each  mission.  This  defines  the  scenario 
number;  intensity  of  fire  support,  artillery,  and  mortars;  available  planning 
time;  nuclear,  biological,  and  smoke  conditions;  engineer,  air  defense,  and 
electronic  warfare  activities  for  both  friendly  and  opposing  forces.  It  also 
tells  visibility  conditions  and  the  number  of  times  the  unit  has  been  through 
the  scenario.  This  data  must  be  manually  entered  by  the  CIS  analyst  and  is 
not  usually  done.  The  working  group  determined  that  these  data  would  be  of 
value,  but  the  group  was  concerned  about  increasing  the  workload  of  the  CIS 
analysts. 


Conclusions 


The  issue  of  the  value  of  data  for  Lessons  Learned  and  a  recognition  of 
the  shortage  of  personnel  and  current  workload  of  OCs  and  CIS  analysts  was  a 
recurring  theme  in  the  working  group  meeting.  Only  a  limited  amount  of 
information  can  be  collected  through  instrumentation.  Much  of  the  data  of 
value  for  Lessons  Learned  requires  systematic  observations  collected  in  a 
consistent  manner.  The  current  software  would  support  much  of  this  data 
^ collection,  but  it  would  be  expensive  in  terms  of  personnel  time.  It  was  the 
consensus  of  the  working  group  that  additional  requirements  could  not  be 
generated  without  additional  resources  being  supplied.  For  example, 
additional  personnel  were  assigned  to  collect  data  on  the  effectiveness  of  the 
Bradley  Fighting  Vehicle.  The  Electronic  Clipboard  will  be  another  valuable 
training  evaluation  and  research  data  gathering  tool. 

Army  decision  makers  need  to  balance  the  potential  utility  pf  the  National 
Training  Center  for  improving^octrine, )vtactics,  training  systems,  -equipment, 
andfcprocedures  against  competing  requirements  for  personnel  resources 
throughout  the  Army.  The  potential  of  the  NTC  for  Lessons  Learned  has  not 
been  realized.  CATA,  ARI  and  other  primary  users  of  the  data  for  Lessons 
Learned  have  utilized  the  available  data  as  effectively  as  possible.  However, 
they  recognize,  and  have  described  in  this  report,  how  much  more  could 
potentially  be  achieved  if  additional  resources  were  available. 
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